As described in the above-referenced co-pending application entitled "Mechanism for Registering Digital Images Obtained from Multiple Sensors Having Diverse Image Collection Geometries", an a priori requirement of the successful exploitation of information contained in multiple images of a scene, such as spaceborne or airborne derived images of the earth, is the mutual registration or alignment of the images. Unfortunately, this requirement is not easily fulfilled when the images to be co-registered are derived from diverse types of sensors, such as those having different wavelength sensitivities and observation and collection geometries. Indeed, it is often the case that multiple images of the same terrestrial area are obtained by the same sensor at different times (seasons) of the year.
Advantageously, the invention described in the above-referenced co-pending application entitled "Mechanism for Registering Digital Images Obtained from Multiple Sensors Having Diverse Image Collection Geometries" successfully addresses this problem by employing an image processing mechanism which correlates the edge content of reduced sized neighborhoods of pixels distributed throughout digital images to be mutually aligned. The digital images upon which the correlation mechanism operates are obtained by projecting or translating originally obtained images to a prescribed registration surface, such as a terrestrial image plane, using geometry projection parameters of the respective image collection systems from which the original images were obtained. Then, based upon a neighborhood correlation of edge content, an estimate is derived of the misregistration of the two digital images in the registration surface. This misregistration estimate is employed to modify either or both of respective sets of parameters through which the collection geometries of the image collection sensors are defined, thereby improving the degree of co-registration of the respective projections of the two original images upon the registration surface. Once the parameters of the geometric models of the image collection sensors have been updated, the neighborhood correlation process is repeated for increasingly higher spatial resolution versions of each of the two images up to and including the original, highest spatial resolution versions of the two terrestrial image plane-referenced images, so as to iteratively refine the geometric models of the sensors and thereby further improve mutual registration of the two images on the registration image surface. Upon completion of this iterative process, the parameters of the geometric models will have been effectively optimized, so that co-registration of images of any of the available spatial resolution images derived from the modelled sensors may be conducted.
The database addresses of the pixels of a respective image to be displayed for co-registration with other images and examination by the user are generated in accordance with operator specified coordinates within the registration surface (e.g. latitude and longitude of the confines of the area of a terrestrial registration surface). Namely, for a given sensor, its associated geometry model transforms pixel coordinates in the digital image space to coordinates within a prescribed co-registration surface. Thus, for the case of a terrestrial image obtained from a satellite sensor, the associated geometry model will effectively translate the respective line and pixel locations of the spatial array of pixels that make up the digital image to corresponding latitude and longitude coordinates on a predefined geographical surface having a prescribed base elevation characteristic (e.g. a planar ground elevation surface).
To facilitate exploitation of the information contained in the images, multiple image files that have been selectively accessed by the user from a workstation database for display and manipulation (e.g. pan or roam, zoom and rotate) are preferably imported into a `window`-based user interface, such as SunOS Open Windows Version 3.0, identified in the above identified co-pending application entitled "Mechanism for Registering Digital Images Obtained from Multiple Sensors Having Diverse Image Collection Geometries", which allows the user to rapidly import and move among the plural images of interest (for example, by means of customarily employed `point and click` selections of screen-based user interface icons).
The refined geometry models of the respective sensors from which the digital images have been derived are then employed to inversely translate operator-specified latitude and longitude coordinates in the registration surface into line and sample coordinates of the images stored within the respective image databases and thereby identify which pixels in a respective image database are to be mapped to the display screen. Mapping from the image database to the display screen is effected by a cascaded mapping function, which cascades the image database-to-registration surface translation of the collection geometry model with a registration surface-to-screen coordinate transform. Through this cascaded mapping function, the pixel values at specified image array locations in the database are coupled to an output display driver for presentation as one of a set of images that are co-registered in the prescribed registration surface for exploitation by the user.
When viewing such a set of co-registered images, the image analyst often desires to manipulate the images, for example, by zooming in on a particular area, rotating the image, or panning (roaming) through the image. Unfortunately, because the image manipulation tools currently available for image processing applications are screen-based mechanisms (customarily employed for graphics display applications), which operate exclusively in the two-dimensional coordinate system of the display screen, they are effectively dissociated from the co-registration surface, and therefore cannot successfully be employed to manipulate co-registered images that have been projected or translated (by means of a collection geometry model) on such a co-registration surface, such as a terrestrial surface.
As a consequence, if the exploiter were to attempt to manipulate images (derived from sensors located in a three-dimensional coordinate system with respect to the object being imaged, such as terrestrial images captured by a satellite or airborne sensor), in a currently active window using screen-based image processing software, he would be limited to manipulating the pixels on the screen in accordance with the two-dimensional coordinate system of the screen itself, and would not achieve the intended manipulation of the image in the registration surface of interest, such as a ground elevation surface in the case of terrestrial images; it is within the registration surface that manipulation of the images must be effected.